Correction of Data for Errors of Measurement 

By W. A. SHEWHART 

Introduction 

EVERY measurement is subject to error. This universally 
accepted truth is the result of every-day experience. From 
the simplest type of measurement, such as determining the length 
of a board with an ordinary tape measure, to the most refined type 
of measurement, such as determining the charge on an electron, 
errors are bound to creep in. 

Now, a manufacturer must constantly make measurements of one 
kind or another in an effort to control his production processes and 
to measure the quality of his finished product in terms of certain of 
its characteristics, but, before he can safely determine the significance 
of observed differences in his production processes or in the quality 
of his product as given by these measurements, he must make allow- 
ance for his errors of measurement; i.e., for the fact that the observed 
differences may be larger or smaller than the true differences. To 
make such allowances for the errors of measurement of any character- 
istic, to find out what the true magnitude of the characteristic most 
probably is, to find out, as it were, what a thing most probably is 
from what it appears to be, presents an endless chain of interesting 
problems to be solved. 

Three important types of problems arising in engineering practice 
are discussed in this paper. They are: 

1.- Error correction of data taken to show the quality of a par- 
ticular lot. 

2. Error correction of data taken periodically to detect significant 
changes in quality of product. 

3. Error correction of data taken to relate observed deviations in 
quality of product to some particular cause. 

The solution of the first one is presented here for the first time. 
The solution of the second has been generalized to include cases not 
previously solvable. All three types of problems are illustrated. 
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PART I 

Type 1 — Error Correction of Data Taken to Show the Quality 
of a Particular Lot 

Let us take a specific problem first. Assume that we have a lot 
consisting of 15,000 transmitters ' and a machine with which to measure 
the efficiency of each instrument. Suppose we make one observation 
on each transmitter — a total of 15,000 measurements. Suppose we 
find, as in the distribution illustrated in Fig. 1, that one measure- 
ment is in the efficiency range —1.75 to —1.50, 17 within the range 
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Fig. 1 — Typical frequency distribution. Chart showing observed number of trans- 
mitters versus efficiency 



— 1.50 to —1.25 units, and so on. The vertical height of a point 
represents the number or frequency of occurrence of observations 
falling within the corresponding interval laid off on the horizontal 
axis of the chart. 

So far so good, but suppose a customer wants to buy these trans- 
mitters. We know that some transmitter which appeared to have an 
efficiency within the range of 1.25 to 1.50 units say, may actually have 
had an efficiency within some other interval. We know too that, 
because of the errors of measurement, the transmitters appear to 
differ more among themselves than they really do. We therefore 

1 Of course, the efficiency of a transmitter does not remain constant during a 
series of tests but these inherent variations in the transmitter may be considered, 
for our purpose, as forming a component part of the resultant error of measurement. 
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desire to find the most probable numbers of transmitters within the 
different intervals indicated in Fig. 1. 

Analytical Statement of Problem 

Let us assume that the most probable number of transmitters within 
the interval of efficiency from X to X-\-dX is fr(X)dX. It is this 
function fr(X) that we want to find. Similarly let us assume that 
there is some function f (X) such that f {X)dX gives the observed 
number of transmitters appearing to have efficiencies within the 
interval X to X-\-dX where the measurements are made by a method 
wherein the probability of making an error within the interval x to x 
-\-dx is f£(x)dx. It is reasonable to expect that, if two of these func- 
tions are known, the third can be easily determined. We shall pro- 
ceed to show that this is the case. Let us first find the law of error 
experimentally. 

Finding the Law of Error 

The problem is to determine the chance of making an error of a 
given magnitude in measuring the efficiency of any transmitter. 
Naturally, the only way of doing this is to make a series of measure- 
ments on a single transmitter from which we can determine the 
observed frequency of occurrence of measurements which differ from 
the average by some fixed amount, and thus find what percentage of 
the total number of measurements may be expected to fall within 
any given range on either side of the average. Common sense and 
intuition may tell us that we may expect to find a large percentage 
of the measurements within a narrow range on either side of the 
average, that there will be just as many measurements greater than 
the average by a certain amount as there are less than the average 
by the same amount, and that large deviations from the average may 
be expected to occur with less frequency than small deviations. 
Suppose we make 500 observations of the efficiency of a single trans- 
mitter and find the distribution given in Fig. 2. Just as we might 
have expected, the observed values of the efficiency of the transmitter 
are grouped symmetrically about the average of all the observed 
values. We see that the maximum deviation between observations 
on a single transmitter is quite large (33%) compared with the actual 
maximum differences observed between the efficiencies of the trans- 
mitters. 

The results reproduced in Fig. 2 suggest that the deviations for the 
case in_hand are distributed in a manner closely approximating the 
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bell-shaped distribution so familiar in the theory of errors. We often 
find, as we do in this case, that the observed distribution can be closely 
approximated by a function Je(x) of the form 

1 (X-X) 1 



/e(x)cIx = 



<tV2tt 



e 2o-2 dx, 



(1) 



where fE{x)dx is the probability that an error x will lie within the 
interval x to x-\-dx, a is the root mean square or standard deviation, 
X is the arithmetic mean value and {X — X) is the deviation x. The 
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Fig. 2 — Typical form of distribution of errors of measurement. Chart showing 
number of measurements on a single transmitter versus efficiency 

function Je(x) is referred to in the literature as the normal law of 
error. If we try to fit such a curve to the deviations 5 given in Fig. 2, 
we obtain the results shown in Fig. 3. This figure is the same as 
Fig. 2 except for the addition of the smooth normal curve of error 
calculated for the observed data. Without further consideration, we 
shall assume the law of error to be normal and hence of the form 
indicated by Equation (1). 



Finding the True Distribution fr(X) 

We have next to consider the choice of the function to represent 

the true distribution fr(X). Often we have reason to believe that this 

2 If the average of the observed values of the 500 observations of efficiency given 
in Fig. 3 is assumed to be the true value of the efficiency of the transmitter, then the 
deviation of an observed value from this mean is also the error of this observed value. 
We shall use the terms "error" and "deviation" interchangeably in this sense. 
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is also approximately normal, and hence we shall consider first the 
method for finding the observed distribution f a (X) for the special case 
when both the true distribution fr(X) and the law of error /e(X) are 
normal; i.e., when they are both of the form given by Equation (1). 

We shall first obtain an experimental answer to this problem. 
Suppose we take, say, 1,000 instruments of some kind which are 
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Fig. 3 — Chart showing the observed distribution of errors fitted by a typical smooth 
curve. Data of Fig. 2 fitted by normal law of error, Eq. 1 



known to be distributed in normal fashion, in respect to some char- 
acteristic, with a standard deviation or- Let us measure each of 
these instruments by a method subject to the normal law of error 
whose standard deviation oe is i or- The results of one such experi- 
ment are given in Fig. 4. The observed frequencies of occurrence 
are represented by the circles. It was found that this observed 
distribution could be closely approximate d by a normal law f (X) 
for which the standard deviation <j was V a\-\- a%. This experiment 
suggests a general theorem which will be demonstrated analytically 
in a succeeding paragraph. The theorem is: When the true distri- 
bution fr(X) and the law of error f E (x) are both normal (hence ex- 
pressible in form indicated by Equation (1)) with root mean square 
or standard deviations o> and &e respectively, the most probable ob- 
served distribution will be normal in form with a standard deviation 

The observed distribution in Fig. 1 is asymmetrical and hence not 
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normal as it should be if fr(X) and fs(x) were both normal. We 
must therefore, try some other function for fr(X). 

Of course, experiments might be performed for other types of true 
and error distributions, but in all such cases the results, as in the 
illustration just considered, would be subject to errors of sampling. 
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Fig. 4 — Experimental results shpwing effects of errors of measurement. Normal curve 
fitted to observed points, when the true distribution and the law of error are both 

normal 



Hence we shall proceed at once to the analytical treatment of the 
problem. 

Assuming the law of error to be normal, we see that the fraction 
fE(x)dx of the number of objects having magnitudes between X-{-x 
and X-\-x-\-dx will be measured with an error between — x and —x — dx 
and hence will be observed as of magnitude X (Fig. 5). Thus 



/„(*)= f~ MX+x)-± ? =e-£-idx. 



(2) 



For the particular case treated in a previous paragraph where both 
the true distribution fr(X) and the law of error }e(x) are normal, 
we may write Equation (2) in the form 



f (X)dX = 



(Tj-ct^Z; 



£ 



(X+x)> 



e 2<j\ e la'k dX dx 



(3) 



where <?t and <*e are the root mean square or standard deviations of 
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the true and error distributions respectively. Integration of Equation 
(3) gives 3 



1 _*i 

fo(X) = j=e 2*i, 



<T 



V2i 



where 



To = Vffr+fli' 



(4) 



(5) 



Equations (4) and (5) are the analytical expression for the rule stated 
previously, for finding the observed distribution f„(X) when both 
the true and error distributions are normal, because Equation (4) 
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Pig 5 — Chart used in explaining the derivation of f„(X) in terms oi f T (X) 

shows it to be normal and Equation (5) expresses the standard devi- 
ation (T of the observed values in terms of those of the true values 
and of the errors. 

In practice, however, we often find that the true distribution is 
non-symmetrical or skew and can be more nearly approximated 
by the function 4 



/m 1 -JLr, krfX Z 3 \"| 



(6) 



where kr is a measure of the asymmetry or skewness, the modal or 



kcr-i 



most probable value of X being at a distance ^ from the average 

3 See Appendix 1 where another method of solution is given. 

4 This is often referred to in the literature of statistics as the second approxima- 
tion. It is in fact the first two terms of the Gram-Charlier series. 
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value of X. Substitution of this expression and a normal error 
function in Equation (2), yields upon integration 5 the following 
distribution f (X) of the observed values 

where 

oo = V o- T -\-a : E , (5) 

and 

ko = kA. (8) 

We see that the distribution fo(X), Equation (7), of the observed 
values is of the same form as that/ r (X), Equation (6), of the true 
values. The standard deviation of the errors of measurement °e, 
as in the previous case, has equal weight with the standard deviation 
0t in influencing the standard deviation <r of the observed values. 
The degree of asymmetry of the observed distribution as measured 
by the skewness k is, however, less (Equation (8) ) than that of the 
true distribution as measured by the skewness kr of the true dis- 
tribution. 

Now we can correct the observed distribution, Fig. 1, for the errors 
of measurement, because we find that the observed frequencies, Fig. 1, 
can be closely approximated by a function of the type defined by 
Equation (7). Knowing that the law of error, Fig. 3, is normal we 
conclude that the true distribution fr(X) must be a function of the 
same type as f (X) was found to be except that the true standard 
deviation o"rwill be, from Equation (5), \/<tI-{-<j e and the true skewness 

kr will be, from Equation (8), -f- k . Now, a and k can be calcu- 

lated from the observed distribution, Fig 1, and °e can be determined 
by the data given in Fig. 3. 

Thus finding the values of <*t and kr and substituting them in 
Equation (G), we have the function fr(X) representing the true dis- 
tribution which we started out to find. From this knowledge of 
fr(X) we can now get the most probable frequencies of occurrence 
of the different efficiencies. Subtracting these frequencies from those 
observed and shown in Fig. 1, we get the corrections plotted in Fig. 6, 
expressed as percentages of the observed frequencies. 

6 This solution is also obtained by another method in Appendix 1. 
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Summary 

We are now in a position to summarize the practical routine to be 
followed in finding the most probable distribution fr(X) of quality 
when the observed distribution is given. 

To find fr(X), we must first know the law of error //•(*)• We 
must show this to be normal and find the standard deviation °e 
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Fig. 6 — Correction which must be applied to the observed distribution of transmitters 
Fig. 1, because of the existence of errors of measurement 

by multiple tests on a single unit. The error made in determining 
the standard deviation o E from n observations is /=-. Hence the pre- 
cision we attain in finding fr(X) depends upon the number of obser- 
vations n made in finding °£. 

Having found *e to the required degree of precision, we must next 
discover whether or not the observed distribution fr{X) is either 
normal or the second approximation. Standard statistical methods 
can be used for this purpose. 

If the/ (A r ) is normal, we then know that 

i x* 

MX)" 



\/2ir(<T--cr- K ) 



e 2w'z-<T'i)- 



and, if / (X) is second approximation, we know that fr(X) is given 
by Equation ((5), where or and kr can be found with the aid of Equa- 
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tions (5) and (8) in terms of the observed values of or, <t<, and k . 
In other words we have 

1 x 

f T (X)=-j= e 2(o*-o%) 






PART II 

Correction of Data Taken Periodically to Detect Significant 
Changes in Quality of Product 

Irrespective of the care taken in defining and controlling the manu- 
facturing processes, the units of a product will differ among them- 
selves in respect to any measurable characteristic. Random fluctua- 
tions in such factors as humidity, temperature, grade of raw material, 
and wear and tear on machinery may produce such differences be- 
tween units of a product. Such random variations in the factors 
underlying the manufacturing process usually yield a product in 
which the units differ in random fashion according to some law of 
probability. 

Customarily, product is inspected periodically, and the data are 
analyzed to determine if the observed difference in two samples is 
greater than can be accounted for as a random variation. If it is, 
we may assume that the manufacturing processes have changed 
significantly for some reason which further investigation should dis- 
close. Now, the presence of errors of measurement effectively in- 
creases the magnitude of the random differences to be expected from 
one sample to another and hence makes it harder for us to detect 
trends or fluctuations in product. Let us investigate this effect of 
errors of measurement. 

Symbolic Statement of Problem 

Symbolically we may assume that the probability of production of 
a unit of product having a characteristic X within any range X to X 
ArdX is ir{X)dX, where the characteristic X is measured by a method 
subject to a law of error /b(*)i so that f E (x)dx represents the proba- 
bility of occurrence of an error x within the range x to x+dx. The 
problem is to find the corresponding distribution f {X) for the ob- 
served magnitudes. 



CORRECTION OF DATA FOR MEASUREMENT 21 

General Solution of Problem 

Obviously the observed magnitude X is the algebraic sum of the 
true value X and the error x. Assuming that there is no correlation 
between these two quantities, the probability of a unit having a value 
of X within the range X to X + dX being measured with an error x 
within the range x to X + dx is f r (X)dX f E {x)dx. Assuming that 
X = X-\-x we may write the probability 

yo=fo{Xo)dX = f MX -x)dX f E {x)dx, 

%* — 00 

because f (X ) is obtained by taking into account that all possible 
values of x between + oc and — <* may be combined with a given X. 
This integral is of the same form as that given in Equation (2). Inte- 
gration for the case where both friX) and fs{x) are normal gives 

1 xl 

fo(X )= 7==e _ ^l 

ffoV 27T 

where as before a = V^t+^b- This result is well known as the law 
of propagation of error. 

When Je(x) is normal and fr(X) is given by the first two terms of 
the Gram-Charlier series, Equation (6), with skewness kr and stand- 
and deviation ct, the observed distribution f (X o ) is of the same func- 
tional form as the true distribution fr(X) and has values of standard 
deviation a and skewness k given by Equations (5) and (8) in Part I. 
This result appears to be new. 

Now for the case where the true distribution fr(X) and the law of 

error //••(.%-) are both second approximation type, the integration is 

somewhat tedious, but we can approach a special case of this problem 

easily from a slightly different angle as indicated in Appendix 2. 

Under certain special conditions therein set forth, the resultant dis 

tribution is also second approximation form with a skewness which 

1 
is less than that of either fr(X) or Je(x) and is equal to '77^ k T when 

k T — k El the standard deviation a being again equal to \/a'^-\-a- E . 

Example of Applications to Determine Most Economical Way of Measur- 
ing Quality 

Let us next consider a very simple method of using the above 
results to indicate the most economical method for determining the 
quality of product with a given degree of precision. 
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What is the most economical way of determining the quality of 
product within some predetermined range X±&X with a known 
probability P, where X is the average quality? Let us assume that: 

a\ =cost of selecting each unit and making it available for measure- 
ment, 

o 2 =cost of making each measurement, 

«i=number of units selected, 

«2 = number of measurements made on each unit, 

oi=standard deviation of the errors of observation. 

0-2 = 07- = standard deviation of the true distribution Jt(X). 

Let us take P = .9973. Then the range X±3<r x includes 99.73 per 
cent, of the observations, and hence AX = 3(Xx- 

The average of w 2 measurements made on one unit is the observed 
value of the magnitude X for that unit, and this average has the 

o"i 

standard deviation <x e = ,— . Hence, from the theory of the preced- 
ing section, the standard deviation of the observation is 



0Q 






The standard deviation of the average of Wi observations is 
ox = —p= and we find upon solving for »i, 

»2 

»l= 5— . 

a 'x 

The cost of inspection is 

y =ai Mi+a 2 Mi «2, 
and by customary methods this can be shown to be a minimum when 

ff 2 A/ fl2 

The following values correspond to one practical case: 
A X = . 3 unit ai = $0.50 

o-i = .3 unit a 2 = $0.02 

o 2 = .9unit P=.9973 
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Thus with the aid of the above theory we find the most economical 
method of inspection requires 2 observations on each of 80 units. 

Application in Setting Limit Lines 

Over 99 per cent, of the averages of samples of size N drawn from 
a product whose law of distribution is fr(X) where fr(X) is either 
normal or second approximation may be expected to lie within the 

limits defined by the true average X plus or minus 3—?=. If an average 

falls outside these limits, this fact is taken as probably indicating the 
existence of a trend or cyclic fluctuation in product, the cause of 
which should be sought. The presence of errors of measurement 
increases the separation of these limits to Gov, from Got. Our pre- 
cision of detecting trend or cyclic fluctuation is thereby decreased. 

Cases often happen in practice where tr is from 15 per cent, to 
25 per cent, greater than or- In some instances a has been found 
to be nearly 50 per cent, greater than or. 

PART III 

Error Correction of Data Taken to Relate Observed 

Deviations in Quality of Product to 

Some Particular Cause 

In many practical cases it is not possible to write down an equation 
to show how the quality of a finished product depends upon the 
factors controlled by different manufacturing steps. To cite one such 
case, we may know that the quality of the finished article depends 
upon the control of the temperature to which some of the piece parts 
are heated in the process of manufacture. Thus the microphonic 
properties of carbon depend upon the temperature to which the 
carbon is heated. In cases where the relationship between quality 
and some factor (such as temperature in the above illustration) can 
only be determined through a study of the correlation existing between 
the quality and the particular factor, use must be made of the correl- 
ation coefficient r which is defined as 

r = 



(T x <TyN 



where x and y represent respectively deviations from the average 
quality X and the average magnitude Y of some factor which is 
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to be controlled by the manufacturing process, and N is the number of 
observations. Now, if errors of observation are made in determining 
x and y, the observed correlation coefficient r Xoyo is known to be given 
by the expression 

(J x Oy 

r Xo yo = r *» (10) 

GxoOyo 

where <Jx =va x '-\-o\ E and tr yo aB V Oy-f-ff^ , 

<Tx E and <Ty E being the root mean square errors of observation of x and 
y respectively. 

Attention is directed to Equation (10) which shows that the observed 
correlation coefficient r Xoyo is always less than the true correlation co- 
efficient r xy irrespective of the number of observations made. Ob- 
viously, this point is of considerable commercial importance as we 
shall now see. 

If the observed correlation is small, we customarily assume that 
there is little need of trying to control the quality X by controlling the 
manufacturing factor Y, whereas this conclusion cannot be justified 
unless it can be shown that the true correlation has not been masked 
by the errors of measurement. 

This point has had to be taken into account in the development 
of machine methods for testing transmitters and receivers, because 
the calibration curves of the machines in terms of ear-voice tests 
depend upon the correlation coefficient. 

Appendix I 

It may be of some interest to certain readers to note that the results 
given in Equations (4) and (7) can also be obtained in the following 
way by the method of moments so often used in statistical investi- 
gations. 

Assuming that fr(X+x) is expansible in terms of a Taylor's series, 
we get 

MX) -MX) +fft(X) +g (^) V W + 

| |(*) , «X)+ . . . . (ii) 

If we substitute a normal form for fr(X) in Equation (11) and 
solve for the moments oif {X), we find that the odd moments are zero 
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and the ratio of the 4th moment to the square of the 2nd is numerically 
3 which indicates thatf (X) is normal in form. 

A similar substitution of the 2nd approximation form for fr(X) in 
Equation (11) yields a distribution f (X) from whose moments we 
deduce Equation (7). Use is made in this proof of the easily demon- 
strated theorem that 

if i < j, where f E is the jth derivative of the normal law function. 

APPENDIX II 

It is well known that the normal law of distribution may result from 
a system of n (w being large) causes each of which produces an incre- 
ment AX measured from some fixed origin with a probability p = l and 
no increment with a probability q = %. Furthermore the second ap- 
proximation may result from a similar system in which p+q and n 
is large. Under such systems of causes, the probabilities of the oc- 
currences of n, » — 1, ... 3, 2, 1, increments are given by the suc- 
cessive terms of the point binomial (p+q)". 

Let us assume that the symbols p T , q T , nr, AJ\T and p E , q E , n E , Ax 
refer to the systems of causes controlling the product and errors 
respectively. The probabilities of observed combinations ?irAX+ 
n E Ax, (n T — l)AX + (n E — l)&x, ... are given by the successive terms 
of the expansion (£r+2r)" r (pE+qE)" E . Now for the special case 
pr=pE = p and AX = Ax we have the resultant probability distribu- 
tion (p+q)" T+ " E with skewness 

g-p 

R — — , 

V pq{nT-\-n E ) 
and standard deviation 

<r = \/pq(nT+n E ). 

Now if p = q, the skewness k is zero and the observed distribution 
is more nearly normal than either component, and its standard devi- 
ation <r is the square root of the sum of the squares of o"r and <r E . 
This result is similar to that given by Equation (4) of this paper. 

We may also consider by this method a case not treated in this 
paper. When the skewness kr of the true values is equal to that 
k E of the law of en or, or, more particularly, when nT = n E = n, pT = 
pE = p, qr : =qi: = q, P = q, we see that the observed distribution is given 



26 BELL SYSTEM TECHNICAL JOURNAL 

by the successive terms of (p-j-q)" and the skewness of the observed 

distribution k u :s ~7~ k, and the standard deviation a is v 2 a\ i.e. the 

1 
observed skewness is only /= times that of either the true distribution 

or the law of error, and the observed standard deviation a is \/2 
times the standard deviation of either of the true or error distributions. 



